A Supermodular Optimization Framework for 
Leader Selection under Link Noise in Linear 

Multi- Agent Systems 



Andrew Clark, Linda Bushnell 1 , and Radha Poovendran 
Department of Electrical Engineering 
University of Washington, Seattle, WA 98195 
{awclark, lb2, rp3}@uw.edu 



Abstract 

In many applications of multi-agent systems (MAS), a set of leader agents acts as a control input to the 
remaining follower agents. In this paper, we introduce an analytical approach to selecting leader agents 
in order to minimize the total mean-square error of the follower agent states from their desired value in 
steady-state in the presence of noisy communication links. We show that the problem of choosing leaders 
in order to minimize this error can be solved using supermodular optimization techniques, leading to 
efficient algorithms that are within a provable bound of the optimum. We formulate two leader selection 
problems within our framework, namely the problem of choosing a fixed number of leaders to minimize 
the error, as well as the problem of choosing the minimum number of leaders to achieve a tolerated 
level of error. We study both leader selection criteria for different scenarios, including MAS with static 
topologies, topologies experiencing random link or node failures, switching topologies, and topologies 
that vary arbitrarily in time due to node mobility. In addition to providing provable bounds for all these 
cases, simulation results demonstrate that our approach outperforms other leader selection methods, such 
as node degree-based and random selection methods, and provides comparable performance to current 
state of the art algorithms. 

1 Corresponding author. Email: lb2@uw.edu 
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I. Introduction 

Multi-agent systems (MAS) consist of networked, autonomous agents, where each agent 
receives inputs from its neighbors, uses the inputs to perform computations and update its 
state information, and broadcasts the resulting information as output to its neighbors. The 
MAS framework has been used to model and analyze man-made systems in a wide variety 
of applications, including the power grid 0"), formations of unmanned vehicles [2], sensor 
networks [3J, and networks of nanoscale devices flU. Natural phenomena, such as flocking in 
birds |0, have also been modeled as MAS. 

An important sub-class of MAS consists of leader-follower systems, in which a set of leader 
agents act as control inputs to the remaining agents [6J. Each follower agent computes its state 
value based on the states of its neighbors, which may include one or more leader agents. Hence, 
controlling the states of the leaders influences the dynamics of the follower agents. A leader- 
follower system can therefore be viewed as a controlled system, in which the follower agents 
act as the plant and the leader agents act as control inputs 0. 

Existing control-theoretic work on leader-follower systems has shown that the performance 
of the system, including the level of error in the follower node states due to link noise (H, flU, 
[10J, depends on which agents act as leaders. Errors in a follower agent's state occur when the 
inputs from its neighbors are corrupted by noise, causing the agent to update its state based 
on incorrect information. The affected agent will broadcast its updated state information to its 
neighbors, which update their states based on the received information, causing state errors to 
propagate through the MAS. The choice of leader agents determines the level of error in the 
follower agent states due to the propagation of leader inputs through noisy communication links. 

In spite of the effect of the choice of leader agents on MAS performance, the design of 
algorithms for selecting agents to act as leaders is currently in its early stages. Since the number of 
possible leader sets grows exponentially in the number of leaders and the total number of agents, 
an exhaustive search over all leader sets is impractical. An analytical approach for choosing 
leaders in order to ensure that the MAS is controllable from its leader agents was introduced 
in ifTTll . This approach, however, does not consider the impact of noise in the communication 
links between agents, leading to deviations from the desired behavior when even a single link 
experiences noise. 
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Leader selection algorithms based on convex optimization have been proposed for static 
networks in order to minimize the error in the follower agent states due to noise in lfT2ll and [|T3l . 
These convex optimization-based algorithms, however, do not provide provable guarantees on 
the optimality of the resulting leader set. Currently there is no analytical framework for leader 
selection for minimizing error due to noise that provides such guarantees. 

In this paper, we present an analytical approach to solve the problem of selecting the leader set 
that minimizes the overall system error, defined as the mean-square error of the follower agent 
states from their desired steady-state value. We formulate the problem of selecting the optimal set 
of leaders as a set optimization problem, and present a solution framework based on supermodular 
optimization. Our framework leads to efficient algorithms that provide provable bounds on the 
gap between the mean-square error resulting from the leader set under our framework and the 
minimum possible error in both static and dynamic networks. We make the following specific 
contributions: 

• We develop a supermodular optimization framework for choosing leaders in a linear MAS 
in order to minimize the sum of the mean- square errors of the follower agent states. 

• We prove that the mean-square error due to link noise is a supermodular function of the set 
of leader agents by observing that the error of each follower agent's state is proportional 
to the commute time of a random walk between the follower agent and the leader sej^J We 
then show that the commute time is a supermodular function of the leader set. 

• We analyze two classes of the leader selection problem within our framework: the problem 
of choosing a fixed number of leaders in order to minimize the error due to noise, and the 
problem of finding the minimum number of leaders needed, as well as the identities of the 
leaders, in order to meet a given error bound. 

• We extend our approach to a broad class of multi-agent system topologies, including systems 
with: (1) static network topology, (2) random link and node failures, (3) switching between 
predefined topologies, and (4) network topologies that vary arbitrarily in time. 

• We compare our results with other leader selection methods, including random heuristics and 
choosing high- and average-degree agents, through simulation and show that our supermod- 

2 The formulation in this paper differs from that of 1141 . which selected leaders based on minimizing the effective resistance 
of an equivalent electrical network. 
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ular approach outperforms both schemes. We also show that the supermodular optimization 
approach provides comparable performance to state-of-the-art methods based on convex 
optimization. 

The paper is organized as follows. In Section [III we review related work on leader-follower 



MAS. Section III states our basic definitions and assumptions and gives background on su- 



permodular functions. In Section IV we formulate the leader selection problem for the case of 
MAS with a static network topology and derive a supermodular optimization solution. In Section 
|V} we analyze leader selection in networks with time-varying, dynamic topologies. Section VI 



evaluates our approach and compares with other widely-used leader selection algorithms through 



a simulation study. Section VII presents our conclusions. 



II. Related Work 

The impact of a given leader set on system performance was considered in [fT5l . where the 
states of the follower agents are treated as the plant to be controlled, while the leader states 
act as control inputs. In the case of linear MAS, the plant dynamics are given by the graph 
Laplacian of the subgraph defined by the follower agents. In |fl5l . it was shown that the leader- 
follower system is controllable from the leader agents if and only if the Laplacian eigenvalues 
are distinct. An alternative condition for controllability, based on the automorphism group of the 
graph, was given in [fl6l . Controllability of MAS with switching network topology was studied 
in ifTTll . Although these studies characterized the controllability of leader-follower systems with 
a given leader set, they have not addressed the question of finding the leader set. 

A systematic framework for leader selection in the absence of noise was developed in ([TT|. 
The authors showed that leaders chosen according to a matching algorithm on the underlying 
communication graph of the MAS satisfy the structural controllability criterion lfT8l . meaning 
that the system is controllable from the leaders for any choice of follower dynamics, except in 
certain pathological cases. The resulting polynomial-time algorithm returns the minimum number 
of leaders, as well as the identities of the leaders, needed to control the system. This approach, 
however, does not consider the effect of errors in the agent states that are introduced by noise 
in the communication links. 

Leader selection in the presence of noise in networks with static topology was considered in 
10. The authors of [9J introduce the network coherence metric, which measures how close the 
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follower agents states are to their desired consensus values, and equals the iJ 2 - n orm of the leader- 
follower system. It is shown that the network coherence is a monotone nonincreasing function 
of the leader set, and a greedy algorithm for maximizing the network coherence is presented. 
While the network coherence is equivalent to the metric we derive for static networks, provable 
bounds on the optimality of the selected leader sets cannot be derived from monotonicity alone. 

In |fT2|. |[T3l . the authors propose a semidefinite programming relaxation of the problem of 
selecting a set of up to k leaders to minimize the H 2 -noxm defined in |9l- These algorithms, 
however, do not provide any guarantees on the optimality of the chosen set of leaders. 

Furthermore, while current approaches consider selecting a set of up to k leaders in static 
networks in order to minimize the error in the agent states, the problem of selecting the minimum- 
size set of leader agents to meet a given bound on the error, as well as leader selection in dynamic 
networks, is not studied in lEfl. Ifl2ll. 1151. 

When the leader set is given, the effect of noise on leader-follower MAS protocols, such as 
consensus, has been studied using a variety of approaches. For a leader-follower system with 
additive link noise, the steady-state error due to noise was shown to be proportional to the 
graph effective resistance between the leader and follower agents in [fT9l . Alternative metrics, 
based on the transfer matrix between the noise inputs and the follower states, were proposed and 
analyzed in jH. In ifTOl . decentralized control for vehicular networks with static topology, single- 
and double-integrator dynamics, and noise in the agent states was considered, and it was proved 
that at least one leader node must be present in the network to achieve stability. In ll20l . existing 
schemes for consensus and vehicle formation control were studied in the if 2 -norm framework. 
While these methods can be used to design and evaluate a leader-follower system with given 
leaders, they do not address the question of selecting a leader set in the presence of noise. 

The commute time of a random walk on a graph, defined as the expected time for a walk 
originating at a node u to reach a node v, has been extensively studied ET1 . In Il22ll . it was shown 
that the graph effective resistance between two nodes u and v is proportional to the commute 
time of a random walk between u and v. To the best of our knowledge, however, our result 
that the commute time between a node u and a set S is a supermodular function of S does not 
appear in the existing literature. 
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III. Preliminaries and Background 

In this section, we give preliminary information on the system model, system error, and super- 
modular functions. 

A. System Model 

We consider a MAS consisting of n agents, indexed by the set V = {1, . . . ,n}. An edge 
exists between agents i and j iff i and j are within communication range of each other. 
Letting E denote the set of edges, the graph structure of the MAS is given by G = (V, E). For 
an agent indexed i, the neighbor set of i, denoted N(i), is defined by N(i) = {j : G E}. 
The degree of % is defined to be the number of its neighbors \N(i)\. It is assumed that the edges 
are undirected and the graph G is connected. 

Each agent % has a time-varying state, denoted #j(t), which may, in different contexts, represent 
i's position, velocity, or sensed measurement. Let x G R n represent the vector of agent states. 
The set of leader agents, denoted S, consists of agents who receive their state values directly 
from the MAS owner. By broadcasting these state values to their one-hop neighbors, the leaders 
influence the dynamics of the follower agents. Without loss of generality, we choose the indices 
such that x(£) = [xy(t) T x;(t) T ] T , where xj(t) and x ; (t) denote the vectors of follower and 
leader states, respectively. 

The goal of the MAS is for the differences between the states of neighboring agents % and j, 
Xi — Xj, to reach desired values, denoted r^, for all G E, so that Xi — xj = r^. The desired 
state x* is defined as the state satisfying x* — x* = for all j G N(i). We assume that is 
known to agents % and j and the MAS owner, and that there exists at least one value of x* such 
that x* - x* = rij for all G E. 

In the noiseless case, in order to reach the desired state x*, the follower agent i G V \ S 
updates its state according to the linear model 



where W is a real-valued weight matrix with nonnegative entries. Furthermore, it is assumed 
that each link is affected by an additive, zero-mean white noise process, denoted €ij(t), 
with autocorrelation function ~E(eij(t)eij(t + r)) = Vij8{r), where S(-) denotes the unit impulse 




j£N(i) 
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function. The noise values on each link are assumed to be independent. This leads to the overall 
linear dynamics of agent i e V \S 



±i(t) 



(1) 



" E W v( X M - X j( t ) - r ij + e ij(*))- 

In order to minimize the effect of noise, it is assumed that the link weights W% are chosen in 
order to generate a best unbiased linear estimate of the leader agent states [fT9l . The link weights 
are therefore chosen as Wij = v~^/Di, where A — Y^jeNU) v ^ ■ A detailed derivation of these 
dynamics is given in Appendix |A} 

Define the elements of the weighted Laplacian matrix L by 



(2) 



which can be further decomposed as 



L 





(i,3) 


G E 


A, 


i = 


-3 


0. 


else 


/ L ff 




V 


V L u 


L u ) 





where Lff and Lfi characterize the impact of the follower and leader agent states, respectively, 
on the follower update dynamics. The dynamics of the follower agent ([I]) for i 6 V \ S can be 
written in terms of L as 

±i(t) = -D; 1 v^{xi{t) - Xj{t) - nj + €ij(t)) 
jeN(i) 

= [ E v * x &) - E v v x &) - E 



Wi(t) 



ieJV(i) 



-i 



LuXiit) -r / 

jeN{i) 



V LijXj(t)+ y~] LijTij I + u>i(t), 



where tOj(i) is a zero-mean white noise process. Define matrix £? to be an (n — \S\) x \E\ 
matrix, with Bu = Lij if edge / is given by for some j E N(i) and Bu = otherwise. 
The dynamics of the follower agents are given in vector form by 

± f (t) = -D] l {LffXf{t) + L/,x,(t) + Br) + w(t), 

where Df is a (n — 15*1) x (n — 15*1) diagonal matrix with A as the i-th diagonal entry. 
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We assume that the leaders maintain a constant state, x;(£) = xj". Based on this assumption, 
the desired state of the followers is defined to be = —L^j(Lfix^ + Br), where we use the 
fact that Ljj exists when G is connected ll23l Lemma 10.36]. 

B. Quantifying System Error 

The mean-square error in the follower agent states due to link noise in steady-state is defined 
as follows. 

Definition 1. Let x^ e R n denote a desired state for the follower agents. The total error of 
the follower agents at time t is defined by E||xj(t) — x^|||. The system error, denoted R(S), is 
defined as 

R(S) = lim E||x/(t) -x*J|2. 

The following theorem gives an explicit formula for R(S) in terms of the matrix L. 

Theorem 1. R(S) is equal to ^Tr(L^j) = | J2 u ev\s where (Ljj) uu denotes the (u, u)- 

entry of Ljj. 

Proof: In the absence of noise, the follower agent dynamics are given by 

x,(f) = -D?{L, f * f (t) + L f tf + Br). (3) 

Since L// is positive definite when G is connected ll23l Lemma 10.36], x*j. = —Ljj(Lfix* + Br) 
is a global asymptotic equilibrium of In the presence of noise, x_f(t) is given by 

X/ (t) = e - D j £ //* X/ (0) - / e~ D f ^-^DfiLfix* +Br)+ / e - - / £ « ( * _T) w(r) dr. (4) 

Jo io 

The first two terms converge to x^, since x^ is a global asymptotic equilibrium of <f3j>. Since w 
is a zero-mean white process, the expected value of the third term of ([4]) is zero by linearity of 
expectation. Thus Hindoo E(x/(t) — x£) = 0, leading to 

i2(5) = limE||x / (t)-x}||l= V E((x u (t) - <) 2 ) = V var(x v (t)), (5) 

uev\s u&v\s 

where ^ follows from the fact that x u (t) — x* u is zero-mean, where x u (t) denotes the state of 
agent u at time t. In [fT9l Section IV-B], it was shown that Hindoo var(x u (t)) = ^(Ljj) uu (see 
Lemma 6] in Appendix A for a proof of this fact), implying that R(S) = \ J2 u ev\s as 
desired. ■ 
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For u G V \ S, let R(S,u) = l(Ljj) uu , so that R(S) = Y^uev\s R ( S ' u )- Note tnat L ff can 
be computed in worst-case 0(n 3 ) time for a given leader set S. 

C. Supermodular Functions 

Let V be a finite set, and let 2 V denote the set of subsets of V. Then a supermodular function 
on V is defined as follows. 

Definition 2. Let f : 2 V — > M, and to 5 C T C V". 77ze function f is supermodular if and only 
if for any v G V \ T, 

f(S) - f(S U {v}) > f(T) - f(T U W). (6) 

Intuitively, this identity implies that adding an element v to a set S results in a larger 
incremental decrease in / than adding v to a superset, T. This can be interpreted as diminishing 
returns from v as the set S grows larger. A function / is submodular if -/ is supermodular. We 
first define the notion of a monotone set function. 

Definition 3. A function f : 2 V — > R is monotone nondecreasing (resp. nonincreasingj if for 

any SCT, f(S) < f(T) (resp. f(S) > f(T)). 

The following two lemmas from ll24ll will be used in our derivations below. 

Lemma 1. Any nonnegative finite weighted sum of supermodular ( resp. submodular) functions 
is supermodular (resp. submodular). 

Lemma 2. Let f : 2 V — > R be a nonincreasing supermodular function, and let c > be a 
constant. Then g(S) = max {/(S 1 ), c} is supermodular. 

A proof of Lemma [2] can be found in Appendix [Bj 

IV. Leader Selection in Static Networks 

In this section, we consider the problem of selecting a leader set in order to minimize the system 
error in static networks, in which the set of links E and the error variances do not change 
over time. We address this problem for two cases. In the first case, no more than k agents can 
act as leaders, and the goal is to choose a set of leaders that minimizes the system error. In 
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the second case, the system error cannot exceed an upper bound a, and the goal is to find the 
minimum number of leaders, as well as the identities of the leader, such that the system error 
is less than or equal to a. In both cases, we construct algorithms for leader selection. 

A. Case I - Choosing up to k Leaders to Minimize System Error 

The problem of choosing a set S of k leaders in order to minimize the system error R(S) is 
given by 

minimize R(S) 

(7) 

s.t. \S\ < k 

Note that problem ([7]) always has at least one feasible solution when k > 1, for example any 
set consisting of a single node. In what follows, we prove that the total system error R(S) is a 
supermodular function of the leader set S, leading to efficient algorithms for approximating the 
optimal solution to ^ up to a provable bound. We first prove that for each agent u, R(S, u) is 
proportional to the commute time of a random walk on the graph G from agent u to the leader 
set S, and then show that the commute time is supermodular. The supermodularity of R(S) 
follows as a corollary. 

Theorem 2. Define a random walk on the graph G starting at node u G V \ S, in which the 
probability of transitioning from node i to node j G N(i), denoted P(i,j), is given by 

/V../) / ); ■ (8) 

The commute time k(S, u), defined as the expected number of steps for the random walk to reach 
the leader set S and return to u, is proportional to R(S,u) = (Ljj) uu . 

The proof of Theorem [2] can be found in the appendix. 

Theorem 3. The commute time k(S,u) is a nonincreasing supermodular Junction of S. 

Proof: The nonincreasing property follows from the fact that, if 5* C T, then any walk that 
reaches S and returns to u has also reached T and returned to u. Thus k(T,u) < k(S,u). 

By Definition [5J k(S, u) is a supermodular function of S if and only if, for any sets A and 
B with A C B and for any j £ B, 

k(A, u) - k(A U {j}, u) > k(B, u) - k(B U {j}, u). (9) 
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Consider the quantity k(A, u) — k(A U {j}, u). Define A' = A U {j}, and define T Au and T A i u 
to be the (random) times for a random walk to reach A (respectively A') and return to u. Then 
by definition, k(A, u) = E(T Au ) and k(A\u) = E(T A > U ). This implies k(A,u) — k(A',u) = 
E(T Au — T A i u ). 

Let Tj (A) denote the event where the random walk reaches node j before any of the nodes in 
A. Further, let hj Au be the time for the random walk to travel from j to A and then to u, while 
hj u is the time to travel directly from j to u. We have 

k(A, u) - k(A', u) = E(T Au - TA' u \Tj(A))Pr(Tj(A)) + E(T Au - T A , 1t |r i (A) c )Pr(r i (A) c ) 
= E(h jAu - h ju )Pr{Tj{A)) 

noting that if the walk reaches A first, then T Au and T A i u are equal. Hence Q becomes 

E{h jAu - h ju )Pr{Ti{A)) > E(h jBu - hj»)PT(r,(B)) (10) 



In order to prove (10) holds, it suffices to prove that hj Au > hjBu and Pr(rj(A)) > Pr(rj(B)). 
Let Wj au denote the event that, after k steps, a random walk starting at j has either not reached 
node a £ A, or has reached a but has not yet reached u. Define Wf Au = (\ a( z A Wf au . Let I(Wj Au ) 
denote the indicator function of the set Wj Au , and note that 

HWfjJ = l[l(Wjaa) (ID 

by the observations above and the definition of indicator functions. hj Au can be rewritten as 

h jAu = mm{k:I(W* Au ) = 0} =: k* 

k* — l k* — l oo 

( = E nwfjj ( = } E J (^) + E J (^) 

fc=0 fc=0 fe=fc* 

oo jj oa / 

= En^) > e n 

fc=0 aeA k=0 \aeA beB\A 

oo oo 

= E II J ( W W = E J (^W = ^ 

fc=o 6es fc=o 

where (a) follows from the definition of I{Wj Au ), (b) follows from the fact that I(Wj Au ) = 
for k > k\ and (c) follows from the fact that (fl aeA W$J n (o^a £ f\ e A W jau- 
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In order to show that Pr(rj(A)) > Pr{jj(B)), first let Tj(v) denote the event that a random 
walk reaches node j ^ v before v. Hence, 



Tj (B) = f| Tj (v) = ( f| r»] n ( PI ^(v)] C f| r,-( 



Tj(-B) C Tj(A) then implies that Pr(rj(B)) < Pr(rj(A)), as desired. Hence (|T0j) holds, thus 
proving that the commute time is supermodular as a function of the leader set S. ■ 
An alternate proof of Theorem [3] can be found in Appendix |Dj 

Corollary 1. R(S) is a nonincreasing supermodular function of the leader set, S. 

Proof: By Theorem[3j k(S, u) is supermodular as a function of S. Since R(S, u) = (Lj 1 )^ 
is proportional to k(S,u), {L~j l ) uu is supermodular as a function of S as well. By Theorem [TJ 
R{S) = Y^ U £V\s (Lf hence R(S) is a sum of supermodular functions. R(S) is therefore 
supermodular by Lemma [TJ ■ 
Corollary [TJ implies that the problem of selecting up to k leader agents in order to minimize 
the system error for a static network (TTJ) is a supermodular optimization problem. Although 
supermodular optimization problems of this form are NP-hard in general, a greedy algorithm 
will return a set S* such that R(S*) is within a factor of (1 — 1/e) of the optimum value, denoted 
R* BH. 

We now present a greedy algorithm for selecting a set of k leaders for a static network 
topology, which is an approximate solution to ([7]). Let S* denote the set of leader agents at the 
2-th iteration of the algorithm. Sq is initialized to 0. At the i-th iteration of the algorithm, the 
element s* e V is found such that {R(S*_ 1 ) — R(S*_ 1 U {s*})} is maximized. S* is then updated 
to (S*_ 1 U {s*}). The algorithm terminates when either R(S*) = R(S* U {j}) for all j, or when 
i = k (i.e., when the number of leaders is equal to k), whichever condition is reached first. A 
pseudocode description of the algorithm is given as algorithm static-A;. 

The following theorem gives a bound on the performance of static-A;, making use of the fact 
that (T7]) is a supermodular optimization problem. 

Theorem 4. Define R ma x by Rmax — maxji?({z}), which is the worst-case error when a single 
agent is chosen as a leader, and let R* be the optimal value of ([7|). Then the algorithm static-k 
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Algorithm static-fc: Algorithm for choosing up to k leaders 



Input: G = (V, E), link error variances 1/y 




Maximum number of leader nodes k 




Output: Set of leader nodes S* 




Initialization: S* 0, i <- 


1 


while i < k 


2 


s* argmax, ens {R(S*) - R(S* U {j})} 


3 


if R(S*)-R(S*U {<})}< 


4 


return S*; exit 


5 


else 


6 


S* ^ S*U{s*} 


7 


i <- i + 1 


8 


end 


9 


end 


10 


return S*; exit 


11 



terminates in polynomial time and returns a set S* satisfying 

R{S*) <\1- j R * + \ R ma, ~ (l - ^ iT + 

Proof: By Theorem 9.3 of [26, Ch III. 3. 9], for any nonnegative monotone nondecreasing 
submodular function f(S), the greedy algorithm returns a set S* satisfying /(S 1 *) > (1 — 1/e)/*, 
where /* is the optimal value of the optimization problem 

maximize f(S) 

V ; (12) 

s.t. 151 < k 

Since R(S) is nonincreasing and supermodular by Corollary[T] the function f(S) = R max — R(S) 
is nonnegative, nondecreasing, and submodular. Maximizing f(S) is thus equivalent to minimiz- 
ing R(S). Hence, the set S* returned by the greedy algorithm satisfies f(S*) > (l — -) /*. 
Substituting the definition of f(S) yields R max — R(S*) > (l — |) (R max — R*) and rearranging 
terms gives the desired result. 

Algorithm static-/c requires k iterations. At each iteration, the function R(S) is evaluated 0(n) 
times. Since each evaluation of R(S) involves inverting the Laplacian matrix, which requires 
0(n 3 ) operations (and can be reduced to 0(n 2 ) operations, with some loss in computation 
accuracy, if the Laplacian matrix is sparse O), the total runtime is 0(kn i ). ■ 

In [|27l . it was shown that no polynomial-time algorithm that improves on the approximation 



bound (l — -) for an arbitrary problem of the for (12) unless P = NP. There may, however, 
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be additional structure to R(S) other than supermodularity that may improve the guarantees of 
Theorem |4} This remains an open problem. 

B. Case II - Choosing the Minimum-Size Leader Set to Achieve an Error Bound 

When the system is required to operate below a given error bound, denoted a, the problem 
of choosing a minimal set of leaders that achieves this bound can be stated as 

minimize \S\ 

(13) 

s.t. R(S) < a 

Note that, for any a > 0, there exists at least one S meeting the condition R(S) < a, namely 
the set S = V. 



The supermodularity of R(S) enables an efficient approximate solution of (13) by a greedy 
algorithm, given as follows. The set of leaders is initialized to Sq = 0. As with the leader 
selection algorithm static-A;, the node s* that maximizes {^(S^) — R{S*_ 1 U {s*})} is added 
at the i-th iteration, so that S* = S*_ x U {s*}. The algorithm terminates when R(S*) < a 
and returns the set S* = S*. A pseudocode description of the algorithm is given as algorithm 
static- a. 



Algorithm static-a: Algorithm for choosing the minimum-size 
set of leaders to achieve an error bound a 



Input: G = (V, E), link error variances My 




Error at termination a 




Output: Set of leader nodes S* 




Initialization: S* 0, error a + 1 


1 


while error > a 


2 


s* <- argmax, ens R(S*) - R(S* U {j})} 


3 


if (R(S*) - R(S* U {s*})) <0 


4 


return S*; exit 


5 


else 


6 


S* <- S* U {s*} 


7 


error R(S*) 


8 


end 


9 


end 


10 


return S*; exit 


11 



The following theorem gives bounds on the optimality of the set S* returned by static-a. 
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Theorem 5. Let k* be the smallest integer such that a set S exists with \S\ = k and R(S) < a 



(i.e., k* is the optimal value of (13)). The algorithm static-a terminates in polynomial time in 



n. If the algorithm terminates after step k, so that \S*\ = k, then 

k ( R 

^ -i . n 1 L max 

F^ i+i °ns(s^ 

holds, where R ma x is as defined in Theorem [?] 

Proof: Theorem 9.4 of [|26l Ch III.3.9] states that the greedy algorithm for solving problems 
of the form 

minimize 9C y \S\ 

1 ' (14) 

s.t. f(S) > A 

where f(S) is a nondecreasing submodular function returns a set S* satisfying 

\s'\ - g \f(V)-f(S k . 1 ) 



where S' is the optimal solution to (14) and S^-i is the set obtained at the (k — l)-th iteration 
of the greedy algorithm. Letting f(S) = R ma x — R(S), we have that f(S) is a nondecreasing 
submodular function of S. Since the greedy algorithm for optimizing f(S) is equivalent to 
optimizing R(S), we have 

In the worst case, the algorithm will not terminate until S — V, i.e., after n iterations. This will 
require 0(n 2 ) evaluations of R, each of which requires 0(n 3 ) computations, for a total runtime 
of 0(n 5 ). ■ 

V. Leader Selection in Dynamic Networks 

Multi-agent systems may undergo changes in topology or link noise characteristics for three 
reasons. First, the agents may experience link or device failures ||28l . Second, the MAS may 
switch between prespecified topologies [|29l . Third, the topology may vary arbitrarily over time 
due to agent mobility; for example, agents may change positions in order to avoid an obstacle, 
affecting both the set of links E and the set of error variances [|2l- Under each case, the 
optimal set of leaders may be different from that of a static network. In this section, we study 
leader selection for each of these dynamic networks. 
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A. Leader Selection Under Random Link Failures 

Random topology changes may occur due to link failures, which are reflected in the weighted 
Laplacian matrix L of Q. Since the set of links may not be known in advance under these 
circumstances, leaders can be selected to minimize the expected system error based on the 
distribution of possible weighted Laplacians. 

Let C denote the set of possible weighted Laplacians. Define n to be a probability distribution 
on C, so that 7r(L) is the probability that the Laplacian is L E C. An example distribution is the 
random link failure model, in which each link in an underlying link set E fails independently 
with equal probability p. The expected system error is defined by 

e 7T (r(S)) = J2R(s\lMl) 1 

Lec 

where R(S\L) denotes the system error when the leader set is S and the Laplacian is L. 

The problems of (a) choosing a set of k leaders to minimize the expected error E 7T (R(S)) 
and (b) choosing the smallest possible leader set S such that the error is within an upper bound 
a are formulated as 



Choosing up to k leaders 

minimize E n (R(S)) 
s.t. \S\ < k 

(a) 



Choosing minimum number of leaders 

minimize 151 

(15) 

s.t. E n (R(S)) < a 



(b) 

In order to solve ( 15 1) and ( 15 3), it is necessary to compute E n (R(S)) for a given distribution 
7r and leader set S. The summation, however, can have up to 2^ possible topologies under the 
random link failure model, making exact computation of E 7r (i?(S')) difficult. We present two 
approaches to approximating E n (R(S)): first, a Monte Carlo approximation that is valid for any 
distribution n, and second, a gradient-based approximation that is valid when the probability of 
link failures is small. 

Monte Carlo Approximation: Under the Monte Carlo approach, a set consisting of M Laplacian 
matrices {Li, . . . , Lm}, each chosen independently according to distribution it, is generated. 
E n (R(S)) is then approximated by R mc (S) = i Y^Zi R ( s \^)- 

Theorem 6. For any e > 0, lim Af ^oo Pr{\R mc {S) - E n {R{S))\ < e) = 1 for every S C V. 
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Proof: For a given S, each R(S\Li) represents an independent sample from the probability 
distribution 7r. Hence, by the weak law of large numbers, YldLi R{S\Li) converges in probability 
to the expected value E 7r (i2(S')). ■ 
Gradient Approximation: Consider the random link failure model, and let G = (V, E), where E 
denotes the set of links that have failed. Define matrix A by 

-i/y 1 , (i,j)eE 

0, else 
Let L f f = L f - A, so that R(S\L) = J2 u ev\s ( L ]))uu- 

Lemma 3. Suppose that links fail independently and randomly with probability p. Let X = 
max(jj) g £ i/y , and let d denote the maximum node degree of the graph G. Define ||A|| 2 to be 
the maximum singular value of A. Then 1 1 A 1 1 2 < 2pdX. 

Proof: Since A is symmetric and positive semidefinite, ||A|| 2 is equal to the maximum 
eigenvalue of A. Let E{ denote the set of edges incident on agent i which fail. Then 

= v ^ ^ X £ 1 = X l^l ^ P dX ' W 

for each i E V. Furthermore, = ~ P^-^- Hence by the Gershgorin Disc Theorem 

ll30ll . the eigenvalues of A must lie in the interval [0, 2pdX). ■ 
Lemma [3] leads to the following gradient approximation for ~E W (R(S)). 



Theorem 7. Let p, d, and X be as defined above, and let 5 = 2pdX. Then 

R(S\G) = Tr(Ljj) < Tr{L]}) + ^'^L (17) 
where A m j n (L//) is the smallest eigenvalue of Lf. 

Proof: R(S) can be written as 

Tr((L ff - A)- 1 ) » Tr{L- f }) + Tr{Lf f *Lj}) 

< Tr(Ljj) + Tr(UA~ 1 U T AUA~ 1 U T ) 

< Tr(Lfj) + supTr([/A- 1 f/ T At/A- 1 f/ T ) 
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where Lf = UAU T is the eigen-decomposition of Lf. The upper bound occurs when A = UVtU 1 
for some positive semidefinite diagonal matrix Vl. Together with Lemma |3j this implies that 

Tr((L f - A)" 1 ) < Tr(L- f }) + Tr(U 'A^QA' 1 ^) 

= Tr(L- f })Tr{U T U A^ttA' 1 ) = Tr(Lj}) +Tr(A- 1 flA~ 1 ) 

nn , T u (n-\S\)6 
< Tr{L-}) + v 



■mm 



Note that the upper bound on R(S) can be used as a worst-case value for R(S) in the presence 
of link failures. Once an appropriate method for computing or estimating E 7r (i?(S')) has been 



chosen, the leader set S can be selected by solving ( 15 1) or ( 15 3). The following lemma can be 



used to derive efficient algorithms for both problems. 

Lemma 4. Under the random link failure model, for any distribution it on C, the function 
E n (R(S)) is supermodular. 

Proof: By definition, E 7r (_R(S')) = J2lcc R{S\L)tt(L). Since the set C of possible weighted 
Laplacians is finite, this is a nonnegative weighted sum of supermodular functions, and hence 
is supermodular by Lemma [T] ■ 
As a corollary to Lemma |4j the problem of choosing a set of k leaders to minimize the expected 



error ( 1 5 1) , and the problem of selecting the minimum number of leaders to meet a bound on 
the expected error (fT5p), can be solved using algorithms analogous to static-A; and static-a 
respectively. The modified algorithms take the distribution n as an additional input parameter, and 
replace line three in both algorithms with s* ^— argmax^vv {E n (R(S*)) — E n (R(S* U {j}))}. 



B. Leader Selection Under Switching Between Predefined Topologies 

A MAS may switch between a set of predefined topologies T = {G\, . . . , Gm}, each with 
different link error variances represented by the corresponding weighted Laplacians L 1; . . . , L M 
(e.g., a set of possible formations) in response to a switching signal from the MAS owner or 
environmental changes ll29l . Leader selection under switching topologies can be divided into 
two cases. In the first case, the set of leaders is updated after each change in topology [13TI . For 
this case, a different set of leaders Si can be selected for each topology Gi using either static-A; 
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(if a fixed number k of leaders is chosen to minimize error) or static-a (if the minimum number 
of leaders is chosen to achieve an error bound a). 

In the second case, the same leader set S is chosen and used for all topologies [17J. Un- 
der this strategy, we consider two possible leader selection metrics, namely the average-case 
error, given as R avg (S) = jjJ2iLiR(S\Li), and the worst-case error, given as R wor st(S) = 
max {R(S\Li) : % — 1, . . . , M}. R avg (S) is a nonnegative weighted sum of supermodular func- 
tions, and hence is supermodular as a function of S by Lemma [T] The problems of selecting 
up to k leaders in order to minimize R a vg(S) and selecting the minimum number of leaders to 
achieve an error bound a can therefore be solved by modified versions of static-/c and static- 
a, respectively. The modified versions of both leader selection algorithms take the topologies 
{Gi, . . . G M } as input, and replace line 3 in both algorithms with s* <— argmaxj e y\s {R avg (S*) 
—R aV g(S* U {j})}. Similarly, the problem of selecting the minimum number of leaders to achieve 
an error bound a can be solved by a modified version of static-a, also taking {G 1; . . . , Gm} as 
input, and with the same substitution at line 3. 

If R W orst(S) is used as a metric, however, note that the maximum of supermodular functions 
is, in general, not supermodular [|32ll . Alternate approaches for leader selection problems are 
given as follows. 

1) Choosing k leaders to minimize worst-case error: The problem of choosing k leaders in 
order to minimize Rworst is stated as 

minimize R WO rst(S) = max i=lv .. jM R(S\Li) 
s.t. 151 < k 



A max, i K iii,-)iL,i 

(18) 



Let S* be the solution to (|18j), and let R* worst = R wors t(S*). R* worst is bounded below by 
and bounded above by R ma x = max i= i t _ t M max Mg y {R({u}\Li)}. As a preliminary, define 

1 - 

i=l 

Note that, by Lemmas [T] and [2j F C (S) is a supermodular function of S. 

An algorithm for approximating S* is as follows. First, select parameters f3 > 1 and 5 > 0. 
The algorithm finds a set S satisfying R(S) < R* worst and \S\ < (3k. Parameter 5 determines the 
convergence speed of the algorithm. Define oP min = and a° max = R ma x- At the j-th iteration, 
let a j = amax " mi " . The goal of the j-th iteration is to determine if there is a set S j such 
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that IS^ I < (3k and R(S 3 \Li) < a 3 for all i = 1, . . . ,M. This is accomplished by solving the 
optimization problem 

minimize \S\ 

(20) 

s.t. F aj (S)<ai 



Since F a j(S) is supermodular, the solution to (20) can be approximated by an algorithm anal 



ogous to static-ct. If the approximate solution to (20), denoted S 3 , satisfies \S 3 \ < (3k, then set 



a Lx = aj 1 and a Ln = a Ll Otherwise, set a 3 max = oPj^ and a 3 min = a 3 . The algorithm 
terminates when \a 3 max — ot 3 min \ < 5 and returns the current set S 3 . A pseudocode description of 
this algorithm is given as algorithm switching- k. 



Algorithm switching-^: Algorithm for selecting up to k leaders 
to minimize worst-case error under switching topologies 
Input: Topologies G\,...,Gm 

t • i • (1) (M) 

Link error variances i/> ■ i/> . 
Maximum number of leaders f3k, threshold S 
Output: Set of leader nodes S* 

Initialization: S* <- 0, j <- 0, a? min <- 0, oP max <- R max 

v-? — np . 
*-max rmn 



l 

a j i mox 2 mm 2 

r <- 0, S j <- 3 

while F a ] (S) < a? 4 

s* <- argmax^gy^ {F aj (S) - F a j (S U {v})} 5 

# : -s- 5 J 'U{<}, + 1 6 

end while 7 

if r > 8 

max y ' ^min mm 

else 10 

cr' ■ a J a J <— 11 

mm ' "mas w mfli 

i «- i + 1 12 

end while 13 

S* <- S^, Return 14 



Since a^ Q;c — a^ in is strictly decreasing as j increases, switching-A; converges. The optimality 
of switching-A; is given by the following theorem. 

Theorem 8. When 5 = -k and (3 satisfies 



> 1 + log (max R({v}\Li)\ j 



switching-k returns a set S* such that maxj {R(S*\Li)} < R* worst and \S*\ < (3k. 
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Proof: The proof follows from the fact that R(S\Li) is a supermodular function for all i 
and Theorem 3 of ll32ll . ■ 
2) Choosing leaders to achieve an error bound: In order to choose a minimum-size set of 
leaders to achieve an error bound a, the following optimization problem must be solved 

minimize \S\ 

(21) 

s.t. R(S\Li)<a Vi = l,...,M 



The following lemma leads to efficient algorithms for solving (21 ). 



Lemma 5. Problem (21) is equivalent to 

minimize \S\ 



(22) 

s.t. F a (S) < a 



where F a (S) is defined as in (19) 



Proof: The proof follows from the facts that the objective functions of (21) and (22) are 
the same, as well as the fact that F a (S) < a if and only if R(S\Lj) < a for all i. ■ 

Since F a (S) is a supermodular function of S, this is a supermodular optimization problem 
similar to ( p"3| ), and hence can be solved by an algorithm analogous to static-a. 

3 ) Leader selection for switching topologies under random agent and link failures: As in 
the static network case, MAS with switching topologies may experience random link or agent 
failures. In this case, the expected values of the average and worst-case system error are of 
interest when selecting the leaders. 

Under random failures, the i-th topology can be represented as a random variable Gj. Let ir 
denote the joint distribution of Gi, . . . , G^, so that n(Gi, . . . , Gm) = Pr(G\ = G\, . . . , Gm = 
Gm), and let n^G) = Pr(Gj = G). Note that the G/s may not be independent; for example, 



under the random failure model of Section V-A the failure of an agent in any topology implies 
failure in all topologies. 

The average-case expected error 'E 7T (R avg (S)) can be further simplified by 

/ M \ M 

E n (R avg ) = E n X^ 5 ^) = irE^WW)' 

\ i=l J i=l 

which follows from linearity of expectation and the fact that, by definition of Hi, E 7r (_R(S'|Lj)) = 
E 7Ti (R(S\L i )). By Lemma |4j ~E Wi (R(S\Li)) is supermodular as a function of S for all i. Hence, 
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E n (R avg (S)) is a nonnegative weighted sum of supermodular functions, and is therefore a 
supermodular function of S by Lemma [T] 

The problem of minimizing E n (R avg (S)) when the number of leaders cannot exceed k can be 
solved using an algorithm analogous to static-fc. Similarly, the problem of finding the smallest 
leader set S that is within an upper bound a on E vr (i? a „ 9 (S')) can be solved using an algorithm 
analogous to static-ct. Both modified algorithms take iti, . . . , ttm as additional inputs and have 
line 3 changed to 

s* <- arg max { ^ E (W) " R ( S U W |L*)) J • 

Considering the expected worst-case error, E n (R worst (S)), observe that, by Jensen's inequal- 
ity 1531. 

eJ max {R{S\Li)} ) > max {E w (i2(5'|L i ))} = max {E Wi (i2(5|Li))}. (23) 

\i=l,...,M I i=l,. ...M i=l,...,M 



The lower bound (23) is the worst-case expected error experienced when the leader set is S. 
Since E 7ri (i2(5|L i )) is supermodular as a function of S, the function 

F C (S) = maxfE^OSIL,))^} (24) 

is supermodular in S by Lemma [2] Therefore, the problem of selecting up to k leaders in order 
to minimize E 7r (i? wors4 (S')) can be approximately solved by an algorithm similar to switching- A; . 
The modified algorithm takes 7Ti, . . . , ttm as additional input, and replaces the function F a j(S) 



at line 5 with the function F a j(S) defined in (24) 



In order to choose the minimum-size set of leaders such that maxj = i ... m {E 7r (-R(5 , |L i ))} is 



below an error bound a, a supermodular optimization problem analogous to (21 ) can be used. 



The constraint of the modified problem is given by F a (S) < a with F a (S) defined as in (24). 



C. Leader Selection Under Arbitrarily Time-Varying Topologies 

In this section, MAS with topologies that vary arbitrarily in time are considered. We assume 
that time is divided into steps, and that at step t, the MAS owner has knowledge of the topologies 
Gi, . . . , Gf-i for steps 1,2, . . . ,t — 1, respectively, as well as their corresponding weighted 
Laplacians L\, . . . , L t _i, but does not know the topology G t for step t. Furthermore, we assume 
that, while the topology may vary arbitrarily over time, the variation is sufficiently slow that the 
agent dynamics approximate the steady-state for graph topology G t during the t-th time step. 
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Under this model, a fixed set of leaders chosen at step t = 1, may give poor performance for 
the subsequent topologies G2, ■ ■ ■ , Gt- Instead, it is assumed that, at step t, a new set of leaders 
St is selected based on the observed topologies G±, .. . ,G t -i- The error for each topology is 
given by R(S t \L t ). The leader selection problem for time- varying topologies is stated as 



minimizes, ,...,s T Tj=i R ( s t\ L t) 
s.t. \S t \ < k 

Problem ( [25] ) is an online supermodular optimization problem fl34). The method for choosing 
a set of leader nodes S% for the t-th time step is as follows. Consider the static-/c algorithm. 
Since G t is unknown and random, the node j that maximizes {R(S* i ) — R{Sti U {j})} at the 
i-th iteration of the algorithm is also random. Let 

tt m (0 = Pr(arg max - R(SU U {j})} = I). (26) 

3 

Then for step t, instead of selecting deterministically as in Line 3 of static- A;, s* ti is selected 



probabilistically with distribution in (26). 

In general, the exact values of n t f will not be known during leader selection. To address this, 
an online learning technique is used to estimate n t based on observations from the previous 
t — 1 time steps. Under this approach, a set of weights w t] i, . . . , Wt,k is maintained, where Wt,i 
is a vector in R n with w t> i{j) representing the weight assigned to choosing node j as the i-th 
leader during step t. Define 

a* 4 exgmax{R(S*\L t ) - R(S* U {j}\L t )}. (27) 



In other words s^f in (|27|) is the best possible choice of s* t ^ for the topology G t . Then define 
the loss l tj ij associated with choosing s* ti = j to be 



R(s:\L t/ 


- r(s; u {j}\L t ) 


R{S* t \L t ) 


- R(S; U {s°f}\L t ) 



At the end of step t, the value of w t i (j) is updated to w m j(j) = /3' t iJ w ti j(j), where (3 E (0, 1] 
is a system parameter that can be tuned to adjust the performance of the learning algorithm. This 
is interpreted as penalizing node j for suboptimal performance during interval t. By decreasing 
the value of (3, nodes experiencing higher losses l t> ij will be much less likely to be selected 
during the (t + l)-th time step. Wt+i^ is then normalized to obtain the estimated distribution 
Kt+i,i' This process is described in detail in algorithm online-A;. 
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Algorithm online-fc: Algorithm for selecting up to k leaders 
for an arbitrarily time-varying topology 
Input: Current weights Wt,i, • ■ • , Wt,k 

Gt = (V, E t ), link error variances i/| • 
Parameter (3 e (0, 1] 
Maximum number of leader nodes k 
Current set of leader nodes 5 t * 
Output: Updated weights Wt+i,i, . . . , ~Wt+i,k 

Set of leader nodes S% +1 
Initialization: 5 t * +1 <- 1 
for i = 1, . . . , k 2 
a% <- argmax, - U {j})} 3 

for j = 1, . . . ,n 4 
, , -i fl(5't%-i)--R(5 , M- 1 u{i}) s 
H,iJ <- «(s; i4 _ 1 )-H(s t v I u{.^ t }) 

wt+i.iO') <- (wt,<O'))0 lMJ 6 
end 7 

7Tt+l,t «- W t+M /l T W t+:M 8 

Choose s*, 1 i randomly with distribution irt+i.i 9 

s; +1 <- 5 t %; u K +1 .J io 

end U_ 

In analyzing this approach, the total error ^ t R(S t \G t ) can be compared to the error achievable 
when all T topologies are known in advance. The following theorem gives a bound on the 
difference between these two errors. 

Theorem 9. Suppose that the algorithm online-k is executed for T steps, and let C7 1; . . . ,Gt 
be the topologies during those steps. Let R ma x be defined as in Theorem |4] Define the error K 
to be 

K 4 (1 - l/e)J2R(S t \L t ) - (max J2(S|L t ) j j . (29) 

Then K < O ( a/ R max kT log n). 

Proof: R(S\Li), . . . , R(S\L T ) is a sequence of supermodular functions bounded above by 
R m ax- Then, by Lemma 4 of [|34l . 



(l-l/e)Y,R((S:\Lt)- (max^i?(S|L t ) }> J < 0(^R max kT log n) (30) 

as desired. 



t=i V 1 1 I t=i 



VI. Simulation Study 

In this section, we evaluate the performance of our leader selection algorithms. Simulations are 
carried out using Matlab. A network of 100 agents is simulated, with agents placed at random 
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positions within a 1000m x 1000m rectangular area. Two agents are assumed to share a link if 
they are deployed within 300m of each other. The error variance v,^ of link is assumed to 
be proportional to the distance between agents i and j. Each data point in the following figures 
represents an ensemble average of 50 trials, unless otherwise indicated. 

For comparison, five different leader selection algorithms are simulated. In the first algorithm, 
a random subset of agents is chosen to act as leaders. In the second algorithm, the k nodes 
with highest degree (i.e., largest number of neighbors) are chosen to act as leaders. In the 
third algorithm, the k nodes with degree closest to the average degree are selected. The fourth 
algorithm simulated, for the case of selecting up to k leaders in a static topology, is the convex 
optimization approach of |[T2l . The fifth algorithm was our supermodular optimization. 



R(S) for different leader selection schemes 



• Random 

— B — Max degree 

— V — Average degree 
-& — Supermodular 
~* — Convex 




Number of leaders needed to achieve error bound 



2 3 4 5 

Number of leader nodes 




0.2 0.4 0.6 o.s 

Normalized system performance 



(a) (6) 

Fig. 1. (a) A comparison of our supermodular optimization approach to leader selection with the convex optimization approach 
of [12]. as well as random and degrcc-bascd leader selection. Either the supermodular or convex optimization approach provides 
minimal error, depending on the number of leader agents, (b) The supermodular optimization approach static-a requires fewer 
leaders to satisfy the error constraint than either the degree-based or random heuristics. Error values are normalized to between 
and 1. 



Case 1: MAS with static network topology - Figure [TJa) compares the performance of the 
five algorithms considered for the problem of choosing up to k leaders in order to minimize the 
total system error. For this comparison, in order to reduce the runtime of the convex optimization 
approach, a smaller network of 25 nodes is used. Figure [TJa) shows the error achieved by the 
different leader selection algorithms for a fixed network topology and varying leader set size, in 
which either the convex or supermodular approaches provide optimal performance depending on 
the number of leader agents. When k = 1,2, 5, 6, our supermodular optimization approach results 
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in lower mean-square error, while the convex optimization approach in |fT2l| selects leaders with 
lower error when k = 3,4. 

For the problem of choosing the minimum number of leaders to achieve an error bound, 
the supermodular optimization approach requires only 40 leaders to achieve normalized error 
of 0.7 (for example), compared to 50 leaders for the random heuristic and over 60 leaders for 
the maximum degree method (Figure [TJb)). Figure [TJb) also suggests that the random heuristic 
consistently outperforms both degree-based algorithms. Selecting the nodes with average degree 
also performs better than selecting the maximum-degree nodes to act as leaders. 



Average R(S) for different network sizes 



Effect of link failures on system error 



■0 — Random 
'B- — Max Degree 
^ — Average Degree 
* — Supermodular 




100 150 200 

Total number of nodes 

fa) 



250 




0.05 0.1 0.15 

Probability of link failures 



0.2 



(b) 



Fig. 2. (a) Error experienced by MAS as the number of agents increases. For each network size, the supermodular optimization 
approach provides the lowest overall error, followed by random selection, average degree-based selection, and maximum degree- 
based selection. Overall error decreases in all cases due to the increase in node density, (b) Effect of random failures on MAS 
error when number of agents is 100 and number of leaders is 10. Although all selection schemes experience an increase in error 
due to link failures, the increase is smallest for the supermodular approach. 



The total system error experienced by the network as a function of network size is explored in 
Figure |2j a). The number of leaders is equal to O.ln, where n is the number of agents. Since the 
deployment area remains constant, adding agents to the network increases the number of links, 
resulting in smaller overall error. Hence, while the supermodular approach still outperforms the 
other methods, the difference in overall error decreases as the node density grows large. 



Case 2: MAS experiencing random link failures - Figure [2Tb) shows the error experienced 



for each method when links fail independently and at random, with probability ranging from 
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to 0.2. The number of leaders is equal to 10 for each scheme, while the network size is 100. The 



supermodular optimization algorithm uses the Monte Carlo approach described in Section V-A 
While each scheme sees a degradation in performance as the probability of failure increases, 
this degradation is minimized by the supermodular optimization method. 



Number of leaders needed for switching topologies 
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Leader selection for switching topologies with random link failures 
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Fig. 3. (a) Number of leaders required to achieve system error of 6 when the MAS switches between M randomly generated 
topologies, where M varies from 1 to 10. The number of leaders needed increases with the number of topologies; the 
supermodular selection method requires the fewest number of leaders, (b) Number of leaders required when the MAS switches 
between M randomly generated topologies and links fail independently with probability p — 0.05. The number of leaders 
required is greater due to link failures. 



Case 3: MAS that switch between predefined topologies - Figure j3|a) shows the number of 
leaders needed to achieve an error level of 6 for each algorithm for MAS under switching 
topologies. For this evaluation, a set of M topologies, where M varied from 1 to 10, is generated 
at random based on the deployment area and node communication range described in the first 
paragraph of Section |VT| A fixed leader set S is then selected using each heuristic. Each scheme 
requires a larger leader set as the number of prespecified topologies increased; however, for the 
supermodular optimization approach, a fixed set of 10 leaders provides an error of less than 6 
for 10 different topologies. Overall, fewer leaders are needed for the supermodular optimization 
approach. Random leader selection requires fewer leaders than the average degree heuristic, 
which in turn outperforms selection of the highest-degree nodes as leaders. 

The case of MAS with switching topologies and independent, random link failures is shown 
in Figure |3^b). Link failures increase the number of leaders required to achieve the error bound 
for each value of the number of topologies, M. While the supermodular optimization approach 
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continued to perform better than the other heuristics in most cases, the performance improvement 
was less significant. 



Average error when topology varies due to agent mobility 
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Fig. 4. Leader selection under arbitrary time- varying topologies. Nodes move according to a group mobility model 1351 with 
speed 30 mis. A new set of leaders is selected every 10 seconds. While the performance of the four selection algorithms is 
comparable, the online supermodular approach online-fc performs better over time by incorporating observed network topology 
information. 



Case 4: MAS with arbitrarily time-varying topology - Figure |4] shows the performance of 
leader selection schemes when the topology varies over time due to agent mobility. Nodes are 
assumed to move according to a group mobility model, in which nodes attempt to maintain their 
positions with respect to a reference point 11351 . The reference point varies according to a random 
walk with speed 30 m/s. Each node's position is equal to its specified position relative to the 
reference plus a uniformly distributed error. A new set of 10 leaders is selected every 10 seconds. 
As in the other cases, the supermodular optimization approach consistently provides the lowest 
error, followed by the random, average degree, and maximum degree heuristics. Moreover, the 
online-A; algorithm improves its performance over time by observing which agents provided the 
best performance when chosen as leaders and assigning those agents a higher weight. 

VII. Conclusions 

In this paper, the problem of selecting leaders in linear multi-agent systems in order to minimize 
error due to communication link noise was studied. We analyzed the total mean-square error 
in the follower agent states, and formulated the problem of selecting up to k leaders in order 
to minimize the error, as well as the problem of selecting the minimum- size set of leaders to 
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achieve a given upper bound on the error. We examined both problems for different cases of 
MAS, including MAS with (a) static network topology, (b) topologies that experience random 
link failures, (c) switching between predefined topologies, and (d) topologies that vary arbitrarily 
over time. We showed that all of these cases can be solved within a supermodular optimization 
framework. We introduced efficient algorithms for selecting a set of leaders that approximates 
the optimum set up to a provable bound for each of the four cases. Our proposed approach was 
evaluated and compared with other methods, including random leader selection, selecting high- 
degree agents as leaders, selecting average-degree agents as leaders, and a convex optimization- 
based approach through a simulation study. Our study showed that supermodular leader selection 
significantly outperformed the random and degree -based leader selection algorithms in static as 
well as dynamic MAS while providing provable bounds on the MAS performance, and provides 
performance comparable to the convex optimization approach. 

References 

[1] S. McArthur, E. Davidson, V. Catterson, A. Dimeas, N. Hatziargyriou, F. Ponci, and T. Funabashi, "Multi-agent systems 

for power engineering applications; Part I: Concepts, approaches, and technical challenges," IEEE Transactions on Power 

Systems, vol. 22, no. 4, pp. 1743 -1752, 2007. 
[2] R. Olfati-Saber, "Flocking for multi-agent dynamic systems: Algorithms and theory," IEEE Transactions on Automatic 

Control, vol. 51, no. 3, pp. 401-420, 2006. 
[3] R. Olfati-Saber and J. Shamma, "Consensus filters for sensor networks and distributed sensor fusion," in 44th IEEE 

Conference on Decision and Control and European Control Conference (CDC-ECC), 2005, pp. 6698-6703. 
[4] A. Galstyan, T. Hogg, and K. Lerman, "Modeling and mathematical analysis of swarms of microscopic robots," in 

Proceedings IEEE Swarm Intelligence Symposium, 2005, pp. 201 - 208. 
[5] C. Reynolds, "Flocks, herds and schools: A distributed behavioral model," in ACM SIGGRAPH Computer Graphics, vol. 21, 

no. 4. ACM, 1987, pp. 25-34. 
[6] W. Ren, R. Beard, and T. McLain, "Coordination variables and consensus building in multiple vehicle systems," Cooperative 

Control, pp. 439-442, 2005. 

[7] M. Ji, A. Muhammad, and M. Egerstedt, "Leader-based multi-agent coordination: Controllability and optimal control," in 

American Control Conference. IEEE, 2006, pp. 1358-1363. 
[8] J. Wang, Y. Tan, and I. Mareels, "Robustness analysis of leader-follower consensus," Journal of Systems Science and 

Complexity, vol. 22, no. 2, pp. 186-206, 2009. 
[9] S. Patterson and B. Bamieh, "Leader selection for optimal network coherence," in 49th IEEE Conference on Decision and 
Control (CDC), 2010, pp. 2692-2697. 
[10] F. Lin, M. Fardad, and M. Jovanovic, "Optimal control of vehicular formations with nearest neighbor interactions," To 

appear in IEEE Transactions on Automatic Control. 
[11] Y. Liu, J. Slotine, and A. Barabasi, "Controllability of complex networks," Nature, vol. 473, no. 7346, pp. 167-173, 2011. 



August 7, 2012 



DRAFT 



30 



[12] M. Fardad, F. Lin, and M. Jovanovic, "Algorithms for leader selection in large dynamical networks: Noise-free leaders," 
in Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference ( CDC-ECC). 
IEEE, 2011. 

[13] F. Lin, M. Fardad, and M. Jovanovic, "Algorithms for leader selection in large dynamical networks: Noise-corrupted 
leaders," in Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference ( CDC- 
ECC). IEEE, 2011. 

[14] A. Clark and R. Poovendran, "A submodular optimization framework for leader selection in linear multi-agent systems," 
in Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference ( CDC-ECC). 
IEEE, 2011. 

[15] H. Tanner, "On the controllability of nearest neighbor interconnections," in 43rd IEEE Conference on Decision and Control 

(CDC), vol. 3. IEEE, 2004, pp. 2467-2472. 
[16] A. Rahmani, M. Ji, M. Mesbahi, and M. Egerstedt, "Controllability of multi-agent systems from a graph-theoretic 

perspective," S1AM Journal on Control and Optimization, vol. 48, no. 1, pp. 162-186, 2009. 
[17] B. Liu, T. Chu, L. Wang, and G. Xie, "Controllability of a leader-follower dynamic network with switching topology," 

IEEE Transactions on Automatic Control, vol. 53, no. 4, pp. 1009-1013, 2008. 
[18] C. Lin, "Structural controllability," IEEE Transactions on Automatic Control, vol. 19, no. 3, pp. 201-208, 1974. 
[19] P. Barooah and J. Hespanha, "Graph effective resistance and distributed control: Spectral properties and applications," in 

45th IEEE Conference on Decision and Control. IEEE, 2006, pp. 3479-3485. 
[20] B. Bamieh, M. Jovanovic, P. Mitra, and S. Patterson, "Coherence in large-scale networks: Dimension-dependent limitations 

of local feedback," To appear in IEEE Transactions on Automatic Control, . 
[21] D. Levin, Y. Peres, and E. Wilmer, Markov Chains and Mixing Times. American Mathematical Society, 2009. 
[22] A. Chandra, P. Raghavan, W. Ruzzo, R. Smolensky, and P. Tiwari, "The electrical resistance of a graph captures its 

commute and cover times," Computational Complexity, vol. 6, no. 4, pp. 312-340, 1996. 
[23] M. Mesbahi and M. Egerstedt, Graph Theoretic Methods in Multiagent Networks. Princeton University Press, 2010. 
[24] S. Fujishige, Submodular Functions and Optimization. Elsevier Science, 2005, vol. 58. 

[25] G. Nemhauser, L. Wolsey, and M. Fisher, "An analysis of approximations for maximizing submodular set functions," 

Mathematical Programming, vol. 14, no. 1, pp. 265-294, 1978. 
[26] L. Wolsey and G. Nemhauser, "Integer and Combinatorial Optimization," 1999. 

[27] G. L. Nemhauser and L. A. Wolsey, "Best algorithms for approximating the maximum of a submodular set function," 

Mathematics of Operations Research, vol. 3, no. 3, pp. 177-188, 1978. 
[28] Y. Hatano and M. Mesbahi, "Agreement over random networks," IEEE Transactions on Automatic Control, vol. 50, no. 11, 

pp. 1867-1872, 2005. 

[29] R. Olfati-Saber and R. Murray, "Consensus problems in networks of agents with switching topology and time-delays," 

IEEE Transactions on Automatic Control, vol. 49, no. 9, pp. 1520-1533, 2004. 
[30] L. Trefethen and D. Bau, Numerical Linear Algebra. Society for Industrial Mathematics, 1997, no. 50. 
[31] M. Mesbahi and F. Hadaegh, "Graphs, matrix inequalities, and switching for the formation flying control of multiple 

spacecraft," in Proceedings of the 1999 American Control Conference, vol. 6, 1999, pp. 4148^1152. 
[32] A. Krause, B. McMahan, C. Guestrin, and A. Gupta, "Selecting observations against adversarial objectives," in Advances 

in Neural Information Processing Systems 20, J. Piatt, D. Roller, Y. Singer, and S. Roweis, Eds. Cambridge, MA: MIT 

Press, 2008, pp. 777-784. 



August 7, 2012 



DRAFT 



31 



[33] A. Leon-Garcia, Probability, Statistics, and Random Processes for Electrical Engineers. Pearson Prentice Hall, 2008. 
[34] M. Streeter and D. Golovin, "An online algorithm for maximizing submodular functions," in Carnegie Mellon University 
Technical Report, 2007. 

[35] X. Hong, M. Gerla, G. Pei, and C. Chiang, "A group mobility model for ad hoc wireless networks," in Proceedings of the 
2nd ACM international workshop on Modeling, analysis and simulation of wireless and mobile systems. ACM, 1999, pp. 
53-60. 

[36] L. Lovasz, "Random walks on graphs: A survey," Combinatorics, Paul Erdos is Eighty, vol. 2, no. 1, pp. 1-46, 1993. 

Appendix A 
Derivation of Agent Dynamics 

In this appendix, the dynamics of each agent are derived following the analysis in Ifl9l , and a 
proof that the error variance is equal to ^tr(Lj^) is provided. The goal of each follower agent i 
is to estimate Xi — x*, the deviation from the desired state. Since x* — x* = r^-, we assume that 
x* = Xj + Tij. Each agent i's relative state estimate for each j 6 N(i) can therefore be written 
as 

Uij %i Xj ~\~ 6{j X{ X^ ~\~ T{j -\- 6{j . 

Letting = y^ — = X; t — x* + e^, the goal of agent i is to estimate Xi — x* given the system 
of equations 

yij = Xi - x* + €ij, jeN(i), 

which can be written in vector form as y = l(xi — x*) + e. By the Gauss-Markov Theorem, 
the best linear unbiased estimator of Xi — x* is equivalent to the least squares estimator, which 
is given by (l T Cl) _1 l T C _1 y, where C is the diagonal covariance matrix of e. The estimator 
reduces to | 

\j&N(i) J jeJV(i) 

thus motivating the choice of link weights D^v^, where Di = J2j^N(i) u ij 1 - ^he following 
lemma, which first appeared in |[T9l , characterizes the asymptotic error variance of the follower 
agent states under these dynamics. 

Lemma 6. Let X{t) denote the covariance matrix o/"xj(t). Then lim^oo X{t) = 

Proof: The dynamics of x^(t) are given by 

x,(t) = -Dj\L ff x f {t) + L f ri + Br) + w(t), 



August 7, 2012 



DRAFT 



32 



where w(t) is a zero-mean white process with variance . Asymptotically, the covariance 
matrix of x/(t) is given as the positive definite solution X to the Lyapunov equation 



/y/.n.v .v /, /7 /V • /y o. 



By inspection, X = \Lf). 



Appendix B 
Proof of Lemma[2] 

Lemma [2] is restated and proved as follows. 

Lemma 7. Let f : 2 V — > R be a nonincreasing supermodular function, and let c > be 
constant. Then g(S) = max {/(S 1 ), c} is supermodular. 

Proof: It is enough to show that for any c, F(S) = max {/(S 1 ), c} is supermodular. The 
proof uses the fact that a function F is supermodular if and only if ([24J) 



F(A) + F{B) < F{A U B) + F(A n B) (31) 
as well as the fact that f(A U B) < (f(A), f(B)) < f(A n B). There are four cases. 



Case 1: a < f(A U B): In this case, (31 ) follows from the supermodularity of / 



Case 2: f(A) < c, f(B) > c: Under this case, (31) is equivalent to f(B) < f(A D B), which 



follows from the monotonicity property. The case where f(A) > c and f(B) < c is similar. 



Case 3: f(A) < c, f(B) < c, f(A f] B) > c: For this case, (31 ) is equivalent to f(A D B) > a, 



which is true by assumption. 



Case 4: f(AnB)<a: Eq. (31 ) is trivially satisfied in this case. 



Appendix C 
Proof of Theorem [2] 

In this appendix, a proof of Theorem [2] is given. Before proving Theorem [2[ the following 
intermediate lemmas are needed. 

Lemma 8. Consider the equation Lv = J. Define v* to be the unique vector satisfying Lv* = J. 
When v* — l, Ji = for i G V \ (S + u), and v * = for s E S, {L^ l ) uu is equal to J^ 1 . 
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Proof: Write v* = [v* f T 0] T and J = [J J J?\ T ■ The equation Lv* = J then reduces to 
LfVj = Jf, which is equivalent to vj = Lj l Jf. Multiplying both sides of the equation by e„, 
where e u has a 1 in the w-th entry and Os elsewhere, yields t>* = e^Lj 1 Jf. The right hand side 
can be expanded to 

(Lj 1 ) ul J 1 H + 1 ) lt ( 71 _|s r |) Jn-\s\ = 1- (32) 



By definition of J, all terms of the left-hand side of (32) are zero except {L^) UU J U . Dividing 
both sides by J u yields the desired result. ■ 

Lemma 9. Let v* be defined as in Lemma [5] Then v* satisfies 

v* = P ^l) v ) e V \ (S + u), (33) 

jeN(i) 

where P(i,j) is defined as in (Is]). 



Proof: By definition of L and the fact that Jj = 0, 



w,j)\v;= (3 4 ) 

y jeN(i) J jeN(i) 



Dividing both sides by J2jeN(i) L (hj) S ives <|33p. 



Lemma 10. Define Vi(S, u) to be the probability that a random walk starting at i with transition 
probabilities given by ([§]) reaches node u e V \ S before any node in the set S. Then Vi = v* 
for all i. 

Proof: By definition, Vi = v* for all i E S and for i — u. It remains to show the result for 
i E V \ (S + u). By the stationarity of the random walk, 

Vi= £ P^,j)vj. (35) 

jeN(i) 

Hence, by (35) and Lemma [9} both v* and v are harmonic functions of the index i. By the 
Maximum Principle [|36l , v* = v. ■ 
Proof of Theorem |2j- By Lemma [8} 

\teN(u) I 
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Hence, it suffices to show that the commute time k(S, u) is proportional to the right-hand side 



of (36). To show this, first observe that the probability Q(u, S) that a random walk originating at 
u will reach S before returning to u is given by 1 — J2teN(u) VtP(u, t), where v is as in Lemma 



10 Proposition 2.3 of 11361 gives Q(u, S) as a function of the commute time, 



Q(u,S) 



2E 



( s ,t)eE v st 



k(S,u)D u 



This yields 



1- £ v t P{u,t) 
teN(u) 



2 J2( s ,t)eE u st 
k(S,u)D u 



Rearranging terms and applying Lemma 10 gives 



k(S,u)= [2 £ 
(s,t)eE 

(s,t)EE 

which proves the theorem. 



D u j 1- Y, v *t p ( u ^) 

t£N(u) 



2 E ) W/ 1 : 

(s,t)e-B 



Appendix D 
Alternate Proof of Theorem [3] 

In this section, an alternative proof to Theorem [3] is presented, which uses an illustrative, 



graphical approach instead of the derivation with indicator functions in Section IV We first 
restate Theorem 

Theorem 10. The commute time k(S,u) is a nonincreasing supermodular function of S. 

Proof: The nonincreasing property follows from the fact that, if S C T, then any random 
walk that reaches S and returns to u has also reached T and returned to u. Thus k(T,u) < 
k(S, u). 

By definition of submodularity ||24||. k(S,u) is a supermodular function of S if and only 
if, for any sets A and B with A C B and any j ^ B, 



k(A,u) - k(A U {j},u) > k(B,u) - k(B U {j},u). 



(37) 
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Consider the quantity k(A,u) — k(A U {j}, u). Define A' = A U {j}, and define T Au and T A i u 
to be the (random) times for a random walk to reach A (respectively A') and return to it. Then 
by definition, k(A, u) = Ei(T Au ) and k(A',u) = Ei(T A / u ). This implies k(A,u) — k(A',u) = 
E(T Au — T A i u ). 

Let Tj(A) denote the event where the random walk reaches node j before any of the 
nodes in A. Further, let hj Au be the time for the random walk to travel from j to A and then to 
u, while hj U is the time to travel directly from j to u. We then have 

k(A, u) - k(A', u) = E(T Au - TA' u \Tj(A))Pr(Tj(A)) + E(T Au - T A , 1t |r i (A) c )Pr(r i (A) c ) 



noting that, if the walk reaches A first, then T Au and T A ' U are equal. Hence (37) is equivalent to 

V(h jAu - hj»)PT(T,(A)) > E(h jBu - h ju )Pr{Ti{B)). (38) 

To complete the proof, it suffices to show that hj Au > hjB U and Pr(rj(A)) > Pr{jj(B)). 
If the walk reaches j before B, then it has also reached j before A, since A C B. Thus 
Tj(B) C Tj{A), and hence Pr{r 3 {B)) < Pr(r 3 (A)). 

It remains to show that hj Au > hjBu- We consider three cases, corresponding to different 
sample paths of the random walk on G. As a preliminary, let ( a b denote the time for a random 
walk starting at a to reach node b. Each case is illustrated by a corresponding figure. 

Case I - The walk reaches v G A before any node in B \ A: In this case, hj Au is equal 
to Q v + ( vu , while hjBu is equal to the same quantity. Thus hj Au = hjBu- This case is illustrated 
in Figure [5] 




Node in A 0%9 Node in B\A 



ONode j (start 
of walk) 



Node u (end 
of walk) 



LEGEND 



Fig. 5. Illustration of the proof that the commute time is supermodular, Case I, showing nodes j, u, and sets A and B with 
A C B. In this case the walk, starting at node j, reaches set A before any node in B \ A and then continues to node u. 
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Case II - The walk reaches t G B\A, then v G A before u: In this case, the hjAu is 
equal to Q v + ( vu = Qt + Ctv + Cvu, since the walk reaches t before v. Similarly, hjBu is equal 
to Q t + Ctu = Q t + Ctv + Cvu, since the walk reaches v after t but before u. Hence hjAu = hjB U 
in Case II as well. This case is illustrated in Figure |6j 




o 



Node j (start 
of walk) 



Node u (end 
of walk) 



LEGEND 



Fig. 6. Illustration of the proof that the commute time is supermodular, Case II, showing nodes j, it, and sets A and B with 
A C B. In this case the walk, starting at node j, reaches set B\ A, then set A, and then node u. 

Case III - The walk reaches t G B\A and then u before v G A: hj^u is equal to Cju + 
Cuv Since the walk reaches t G B \A before u, this is equal to Qt + Ctu + Cuv- However, since 
hjBu is the time for the walk to reach any node in B and then travel to u, hj Bu = Qt + Ctu- 
Thus hjAu > hjBu in Case III. Case III is illustrated in Figure |7J 
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if walk) 



Node u (end 
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Fig. 7. Illustration of the proof that the commute time is supermodular, Case III, showing nodes j, u, and sets A and B with 
A C B. In this case the walk, starting at node j, reaches set B \ A and node u before reaching set A. 



Together, these cases imply (38), and hence the supermodularity of k(S,u) as a function of 



S. 
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